Technical Field
This disclosure relates to animating a virtual character.
Description of Related Art
Animating the communications of a virtual character may integrate lip sync movements and nonverbal movements (e.g., facial expressions and head movement). Several approaches extrapolate such movements by spotting specific words, so called keywords, in the text that a character will speak and use a database of keyword-to-behavior rules to select the virtual character's nonverbal movements. For example, a Behavior Expression Animation Toolkit (BEAT) analyzes the text that the virtual character is to speak with user provided rules to detect keywords and phrases. Then, the program automatically generates the speech part of a virtual character, and the associated nonverbal behavior and facial expression given raw text utterances. In another example, an Articulated Communicator Engine (ACE) focuses on generating specific kinds of hand gestures (e.g., deictic and iconic gestures) during speech. The system scans the text that the virtual character is to speak and looks for specific key words in order to display gestures that the designer of the system has associated with those keywords. In yet another example, a Nonverbal Behavior Generator (NVBG) (similarly detects key words in the text to generate gestures, head movements and facial expressions. Keyword approaches, like those in the foregoing examples, may be limited to detecting specific words.
Another approach to animating a virtual character with lip sync movements and nonverbal movements is to analyze the audio signal of the utterance to determine rhythmic features in the audio and then map those rhythmic features to nonverbal movements. For example, detection of prosodic features of the dialog (e.g., loudness) has been used to generate head movement. In another example, audio features have been used to drive gesturing for a subclass of gestures that are rhythmic in nature, so called beat gestures. These approaches to integrate nonverbal movements based on rhythmic features, along with the keyword approaches above, may be limited in that they generally generate facial expressions and head movements without significant value placed on nonverbal communication with the rest of the body (e.g., hand movements and upper body leaning).